{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/DeepLabCut/DeepLabCut/blob/master/examples/COLAB/COLAB_DEMO_mouse_openfield.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TGChzLdc-lUJ"
   },
   "source": [
    "# DeepLabCut on Single Mouse Data Demo\n",
    "\n",
    "Some useful links:\n",
    "\n",
    "- [DeepLabCut's GitHub: github.com/DeepLabCut/DeepLabCut](https://github.com/DeepLabCut/DeepLabCut)\n",
    "- [DeepLabCut's Documentation: User Guide for Single Animal projects](https://deeplabcut.github.io/DeepLabCut/docs/standardDeepLabCut_UserGuide.html)\n",
    "\n",
    "![alt text](https://images.squarespace-cdn.com/content/v1/57f6d51c9f74566f55ecf271/1559935526258-KFYZC8BDHK01ZIDPNVIX/mouse_skel_trail.gif?format=450w)\n",
    "\n",
    "Demo supporting: Nath\\*, Mathis\\* et al. *Using DeepLabCut for markerless3D  pose estimation during behavior across species. Nature Protocols, 2019 \n",
    "\n",
    "This notebook demonstrates the necessary steps to use DeepLabCut on our demo data. We provide a sub-set of the mouse data from Mathis et al, 2018 Nature Neuroscience.\n",
    "\n",
    "This demo notebook mostly shows the most simple code to train and evaluate your model, but many of the functions have additional features, so please check out the overview & the protocol paper!\n",
    "\n",
    "This notebook illustrates how to use the cloud to:\n",
    "\n",
    "- load demo data\n",
    "- create a training set\n",
    "- train a network\n",
    "- evaluate a network\n",
    "- analyze a novel video"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "txoddlM8hLKm"
   },
   "source": [
    "## Installation\n",
    "\n",
    "### First, go to \"Runtime\" ->\"change runtime type\"->select \"Python3\", and then select \"GPU\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Ew6r4hotoQjt"
   },
   "outputs": [],
   "source": [
    "# Clone the entire deeplabcut repo so we can use the demo data:\n",
    "!git clone -l -s https://github.com/DeepLabCut/DeepLabCut.git cloned-DLC-repo\n",
    "%cd cloned-DLC-repo\n",
    "!ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "yDaY78dFoxyD"
   },
   "outputs": [],
   "source": [
    "%cd /content/cloned-DLC-repo/examples/openfield-Pranav-2018-10-30\n",
    "!ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "q23BzhA6CXxu"
   },
   "outputs": [],
   "source": [
    "# Install the latest DeepLabCut version (this will take a few minutes to install all the dependencies!)\n",
    "%cd /content/cloned-DLC-repo/\n",
    "%pip install \".\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XymV_Hnlp1OJ"
   },
   "source": [
    "### PLEASE, click \"restart runtime\" from the output above before proceeding!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "sXufoX6INe6w"
   },
   "outputs": [],
   "source": [
    "import deeplabcut"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Z7ZlDr3wV4D1"
   },
   "outputs": [],
   "source": [
    "# Create a path variable that links to the config file:\n",
    "path_config_file = '/content/cloned-DLC-repo/examples/openfield-Pranav-2018-10-30/config.yaml'\n",
    "\n",
    "# Loading example data set:\n",
    "deeplabcut.load_demo_data(path_config_file)\n",
    "\n",
    "# Automatically update some hyperparameters for training, \n",
    "# here rotations to +/- 180 degrees. This can be helpful for optimizing performance. \n",
    "# see Primer -- Mathis et al. Neuron 2020\n",
    "from deeplabcut.core.config import read_config_as_dict\n",
    "import deeplabcut.pose_estimation_pytorch as dlc_torch\n",
    "\n",
    "loader = dlc_torch.DLCLoader(\n",
    "    config=path_config_file,  \n",
    "    trainset_index=0,\n",
    "    shuffle=1,\n",
    ")\n",
    "\n",
    "# Get the pytorch config path \n",
    "pytorch_config_path = loader.model_folder / \"pytorch_config.yaml\"\n",
    "\n",
    "model_cfg = read_config_as_dict(pytorch_config_path)\n",
    "model_cfg['data'][\"train\"][\"affine\"][\"rotation\"]=180\n",
    "\n",
    "# Save the modified config\n",
    "dlc_torch.config.write_config(pytorch_config_path,model_cfg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "c4FczXGDoEJU"
   },
   "source": [
    "## Start training:\n",
    "This function trains the network for a specific shuffle of the training dataset. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "_pOvDq_2oEJW"
   },
   "outputs": [],
   "source": [
    "# Let's also change the display and save_epochs just in case Colab takes away\n",
    "# the GPU... If that happens, you can reload from a saved point using the\n",
    "# `snapshot_path` argument to `deeplabcut.train_network`:\n",
    "#   deeplabcut.train_network(..., snapshot_path=\"/content/.../snapshot-050.pt\")\n",
    "\n",
    "# Typically, you want to train to ~200 epochs. We set the batch size to 8 to\n",
    "# utilize the GPU's capabilities.\n",
    "\n",
    "# More info and there are more things you can set:\n",
    "#   https://deeplabcut.github.io/DeepLabCut/docs/standardDeepLabCut_UserGuide.html#g-train-the-network\n",
    "\n",
    "deeplabcut.train_network(\n",
    "    path_config_file,\n",
    "    shuffle=1,\n",
    "    save_epochs=5,\n",
    "    epochs=200,\n",
    "    batch_size=8,\n",
    ")\n",
    "\n",
    "# This will run until you stop it (CTRL+C), or hit \"STOP\" icon, or when it hits the end."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RiDwIVf5-3H_"
   },
   "source": [
    "We recommend you run this for ~100 epochs, just as a demo. This should take around 15 minutes. Note, that **when you hit \"STOP\" you will get a `KeyboardInterrupt` \"error\"! No worries! :)**\n",
    "\n",
    "A new snapshot is saved every `save_epochs` epochs. So once you hit 80 epochs, your latest snapshot in `/content/cloned-DLC-repo/examples/openfield-Pranav-2018-10-30/dlc-models-pytorch/iteration-0/openfieldOct30-trainset95shuffle1/train` should be `snapshot-80.pt`. The best snapshot evaluated during training is saved, and is named `snapshot-best-XX.pt`, where `XX` is the number of epochs the model was trained with."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xZygsb2DoEJc"
   },
   "source": [
    "## Start evaluating:\n",
    "This function evaluates a trained model for a specific shuffle/shuffles at a particular state or all the states on the data set (images)\n",
    "and stores the results as .csv file in a subdirectory under **evaluation-results**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "nv4zlbrnoEJg"
   },
   "outputs": [],
   "source": [
    "deeplabcut.evaluate_network(path_config_file, plotting=True)\n",
    "\n",
    "# Here you want to see a low pixel error! Of course, it can only be as\n",
    "# good as the labeler, so be sure your labels are good!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oxy5JG-kYKF4"
   },
   "source": [
    "**Check the images**:\n",
    "\n",
    "You can go look in the newly created `\"evaluation-results-pytorch\"` folder at the images. At around 100 epochs, the error is ~3 pixels (but this can vary on how your demo data was split for training)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OVFLSKKfoEJk"
   },
   "source": [
    "## Start Analyzing videos: \n",
    "This function analyzes the new video. The user can choose the best model from the evaluation results and specify the correct snapshot index for the variable **snapshotindex** in the **config.yaml** file. Otherwise, by default the most recent snapshot is used to analyse the video.\n",
    "\n",
    "The results are stored in hd5 file in the same directory where the video resides. \n",
    "\n",
    "**On the demo data, this should take around ~ 90 seconds! (The demo frames are 640x480, which should run around 25 FPS on the google-provided T4 GPU)**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Y_LZiS_0oEJl"
   },
   "outputs": [],
   "source": [
    "# Enter the list of videos to analyze.\n",
    "videofile_path = [\"/content/cloned-DLC-repo/examples/openfield-Pranav-2018-10-30/videos/m3v1mp4.mp4\"]\n",
    "deeplabcut.analyze_videos(path_config_file, videofile_path, videotype=\".mp4\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pCrUvQIvoEKD"
   },
   "source": [
    "## Create labeled video:\n",
    "This function is for visualization purpose and can be used to create a video in .mp4 format with labels predicted by the network. This video is saved in the same directory where the original video resides. This should run around 215 FPS on the demo video!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "6aDF7Q7KoEKE"
   },
   "outputs": [],
   "source": [
    "deeplabcut.create_labeled_video(path_config_file, videofile_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8GTiuJESoEKH"
   },
   "source": [
    "## Plot the trajectories of the analyzed videos:\n",
    "This function plots the trajectories of all the body parts across the entire video. Each body part is identified by a unique color."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "gX21zZbXoEKJ"
   },
   "outputs": [],
   "source": [
    "deeplabcut.plot_trajectories(path_config_file, videofile_path)"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "Colab_DEMO_mouse_openfield.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
